In order to solve the problem of low matching quality of the traditional feature extraction and matching algorithm in Unmanned Aerial Vehicle （UAV） visual localization， a new image matching algorithm based on transmission tower area extraction was proposed. Firstly， the image was divided into several overlapping grid areas， and the feature points were extracted by a two-layer pyramid structure for each area to ensure the uniform distribution of feature points. Then， the Line Segment Detector （LSD） algorithm was used to extract the lines in the images， the transmission tower support areas were extracted on the basis of special structure of transmission tower. Finally， the feature points in the transmission tower areas and the background areas were matched respectively in continuous images to further estimate the camera motion. In the rotation and translation estimation experiment， compared with the traditional Oriented Features from Accelerated Segment Test（FAST） and Rotated Binary Robust Independent Elementary Features （BRIEF） （ORB） feature extraction and matching algorithm， the proposed algorithm has the feature matching accuracy improved by 10.1 percentage points， the mean value of relative pose error reduced by 0.049. In the UAV inspection experiment， the relative error of the UAV trajectory estimation by using the proposed algorithm is 2.89%， which indicates that the proposed algorithm can achieve the robust and accurate estimation of the UAV’s pose during the real-time flying around the tower.